1. Field of the Invention
The present invention relates generally to airborne mapping systems and methods and, more particularly, to a system for collecting, processing, displaying, and exploiting aerial imagery with enhanced Three-Dimensional (3D) & Near-Infra Red (NIR) imaging capability.
2. Description of the Background
Aerial photography involves taking photographs of the ground from an airborne platform, typically from camera(s) mounted on aircraft. Aerial photography is commonly used in cartography, including photogrammetric surveys. The typical process employs flying a pattern over a defined area and using one or more cameras on the aircraft to take periodic photos, which are later merged into a completed mosaic aerial survey of the entire area. Aerial photography mosaics have been useful for decades, and there have been many efforts to automate the collection process dating to before the advent of digital photography. For example, European Patent EP0053640 by Fujimoto (Interval Eizo Yugen Kaisha) filed Dec. 4, 1980 discloses a computerized control for aerial film photography that automatically triggers the camera shutters to ensure full coverage of the area being photographed. The inputs to the computer are of aircraft speed, altitude, percentage overlap between successive pictures, and lens angle.
Most airborne imaging platforms are still very expensive, require significant lead-time, stabilized camera gimbals and/or Inertial Measurement Units (IMUs), and still lack the spatial resolution for resolving small features. However, digital cameras have evolved and it is now possible to acquire large-scale imagery with inexpensive consumer-grade equipment, at a significantly lower cost, and a higher spatial resolution. Digital photography also made it possible to mosaic images based on common features in the images. For example, United States Patent Application 20050063608 by Clarke et al. (Epson) published Mar. 24, 2005 shows a system and method for creating a panorama image from a plurality of source images by registering adjoining pairs of images in the series based on common features.
With the subsequent advent of GPS positioning, it became possible to georeference images to ground coordinates. Multiple ground images could be mosaicked into a wide area image based on geotagged GPS coordinates in each image and thereby registered onto a uniform “orthophotograph”, an aerial photograph geometrically corrected or “orthorectified” such that the scale is uniform. Unfortunately, GPS coordinates alone do not provide sufficient accuracy. More information is needed. Indeed, in order to form a precise registration of images it is also necessary to adjust the individual images for topographic relief, lens distortion, camera tilt, etc. An orthophotograph includes such adjustments and can be used to measure true distances as if on a map. U.S. Pat. No. 7,639,897 to Gennetten et al. (Hewlett Packard) issued Dec. 29, 2009 shows an airborne imaging system in which a camera is swept once over a field of view to construct a video mosaic which is used to select settings for focus, exposure, or both to be used and to compute the number and locations of a set of component photographs that will tile the scene. The system then guides the user to sweep field of view of the camera over the scene a second time, visiting each component photograph location.
Orthophotographs are commonly used by a Geographic Information Systems (GIS), which provides a foundation for photogrammetric analyses of the aerial photographs. Photogrammetry is used in different fields, such as topographic mapping, architecture, engineering, manufacturing, quality control, police investigation, and geology. Orthophotographs can be registered to a two-dimensional scale, or in three dimensions. For example, U.S. Pat. No. 7,751,651 to Oldroyd (Boeing) issued Jul. 6, 2010 shows a processing architecture where a camera image is registered with a synthetic 3D model of a scene by combining a geocoded reference Digital Elevation Model (DEM) and a geocoded reference image such that each pixel of the geocoded reference image is associated with an elevation from the DEM. United States Patent Application 20120105581 to Berestov (Sony) published May 3, 2012 shows a method for converting two dimensional images to three dimensional images using Global Positioning System (GPS) data and Digital Surface Models (DSMs). The DSMs and GPS data are used to position a virtual camera. The distance between the virtual camera and the DSM is used to reconstruct a depth map. The depth map and two dimensional image are used to render a three dimensional image.
Multispectral and hyperspectral imaging sensors are able to view different bands in various regions of the electromagnetic spectrum. For example, U.S. Pat. No. 5,999,650 to Ligon issued Dec. 7, 1999 shows a system for generating color images of the Earth's surface based on its measured red and near infra-red radiation. The system classifies each area of land based on satellite-measured red and near-infrared radiation from each area of the land, then associates a color with each land class, and finally colors the image pixel corresponding to each area of the land with the color associated with its class. Multispectral and hyperspectral imaging are especially useful in diagnosing crop health.
A more sophisticated 3D imaging technique, called stereophotogrammetry, involves estimating the three-dimensional coordinates of points on an object. These are determined by measurements made in two or more photographic images taken from different positions. U.S. Pat. No. 5,606,627 to Kuo (Eotek Inc.) issued Feb. 25, 1997 discloses an automated analytic stereo comparator that extracts elevation data from a pair of stereo images with two corresponding sets of airborne control data associated with each image of the stereo image pair. The topographic elevation of each feature is derived from object-space parallax, a base length, and the altitude of a camera station. U.S. Pat. No. 7,508,972 to Maruya (NEC) issued Mar. 24, 2009 shows topographic measurement using stereoscopic picture frames. The picture frames are combined to produce a number of pairs of frames which constitute a stereoscopic image of the target area. Each frame pair is analyzed according to a number of visual characteristics and evaluated with a set of fitness values representative of the degrees of fitness of the frame pair to topographic measurement of the target area. A total of the fitness values is obtained from each frame pair and compared with the total values of other frame pairs. A parallax between the best pair frames is determined to produce first and second sets of line-of-sight vectors for conversion to topographic data.
Aerial photography has proven especially useful for monitoring growth and development of crops and obtaining early estimates of final crop yield. With the use of near infrared aerial photographs, plant physiological and morphological differences can be distinguished within fields and areas of possible plant disease can be identified. Crop Producers can evaluate site-specific yield potential, irrigation efficiency, nitrogen levels and seeding, and can generally improve profitability through yield increases and material savings. Stereophotogrammetry enables further exploitation by providing information about the heights of objects standing off the ground. Such data is useful in all application fields, including agriculture, forestry, mapping and surveying.
Despite piecemeal technology advancements as described above, there remains a need for a turnkey system for guided geospatial image capture from Unmanned Aerial Vehicles (UAVs) and/or manned aircraft, registration and mosaicking, that employs low cost hardware and is highly automated, and that generates geo-referenced Orthomosaics to be rendered in two or three dimensions with the attendant capability to measure the heights of objects.